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Vector  Spaces  and  Linear  Inequalities 

§ 1 .   Introduction . 

The  primary  aim  of  this  paper  is  to  provide  a  simple  and 
coherent  development  of  the  principal  theorems  concerning  sys- 
tems of  linear  inequalities  in  terms  of  the  machinery  of  the 
theory  of  linear  vector  spaces.   At  its  inception  this  develop- 
ment is  based  on  a  simple  geometric  separation  theorem  which  is 
easily  deduced  from  the  standard  theorem  concerning  the  separa- 
tion of  c].osed  convex  bodies.   This  same  geometric  theorem  also 
yields  easily  the  existence  of  completely  mixed  strategies  for 
the  essential  part  of  a  rectangular  game. 

§ 2 .   A  separation  theorem  for  cones . 

In  order  to  prove  the  main  resul.t,  it  will  be  best  to  in- 
troduce notation  and  certain  conventions. 

1.  All  sets  considered  will  lie  in  n-dlmensional  Euclidean 
space  e'^. 

2.  If  K  is  a  set,  then  I(K)  will  denote  the  interior  of  K. 

3.  The  first  orthant  of  E^"^  is  tlie  set  of  all  vectors 

(x, ,...,x  )  such  that  x.  >  0,  1  =  l,...,n.   The  first  orthant 
will  be  denoted  by  O. 

\\.      If  C  and  K  are  two  sets,  then  {a,x)  =  b  will  be  called  a 

separating  hyperplane  of  C  and  K,  if  (a,x)  >  b  for  all  x  e  C  and 

(a,x)  <  b  for  all  x  f.  K.  If  C  and  K  have  a  separating  hyperplane 

then  we  say  that  C  and  K  can  be  separated. 

^■'e  will  also  assiame  the  following  well  toown  theorem: 


:::.j-     rj. 


Theorem  1.   If  C  aiid  K  are  disjoint  and  convex,  then  C  and 
K  can  be  separated. 

Corollary  1.   If  C  and  K  are  convex  and  intersect  only  in 
common  boundary  points,  and  if  I(K)  -^   0,  then  C  and  K  can  be 
separated. 

Proof:   C  and  I(K)  can  be  separated,  and  hence  the  same 
hyperplane  separate  C  and  K. 

i''e  now  present  the  main  result  of  this  section. 

Theorem  2.   Let  C  be  a  closed  convex  cone  such  that 
C  n  0  -  U.   Then  there  exists  a  hyperplane  (a,x)  =  0  such  that 
(a,x)  <  0  for  x  £  C  and  (a,x)  >  0  for  x  £  &,    x  ^   0. 

F_roof:   Let  e  "^  (j  =  l,...,n)  be  the  j-th  unit  vector  in 
e'^.   Let  e"^(e)  =  (-e,-e,...,  1  ,...,-£ ).   X-'e  now  denote  the  cone 

generated  by  e'^(e),  (j  =  l,...,n)  as  (?(e).   '''e  will  now  show  thai 
if  0  <  e  <  l/n-1,  then  C+(  e )  r:^  (T. 


For  e'^'(e)  =  e^   -   e  ][~   e^  =  (l+e)e^'  -  e  >   e'^   .   Thus 

Wl  1^  =  1 


k'  k         k       ^         ^  k 

hence 

or 

Hence  the  e -^ '  s  are  positive  linear  combinations  of  the  e'^(e)*s, 
i.e.  0(e)  Z^'  &. 

It  also  follows  that  if  l/n-1  >   €   >  r.    >   0 ,    then 
(fie)   ~?  0'('/)  .   For 


ill-     .10"-. 


k 
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Thus  e^(>^)  =  ehe)    +   (e-''l)J"  e^-   Thus  eH)])    is  a  non-negative 

linear  combination  of  vectors  in  <!'(e),  i.e.  O'ie)    z:  &{/'()' 

We  now  want  to  show  that  there  exists  €  >  0,  such  that 
— > 
Cit)    I]  C   =   0.   Suppose  this  were  not  the  case.   Let  e.  — >  0  be 

a  monotonic  sequence  where  l/n-1  ^  e*  >  0.   Then  G{  e .  )    HC  con- 

tains  a  vector  x^   ^   0.   Thus  Gf[t.)    0  C  contains  x.  =  x./||x.  ||. 

But  jx.  V  is  a  bounded  sequence  hence  it  contains  a  converging 


-^ 


subsequence    -^x.     (,  -*.  x"  7^   0.      Thus   x"  £  C    and  x"  £    C(  e .  )    for 

*    ^k!  ,.  CD  ^, 

each  1.      This    iraplies   that   x"  (-.     0    0'(e.)   =   ^' .      Thus   x"  £    0  OC 

i=l  ^  ., 

which   is    a   contradiction.      Henco    there   exists    e"   such   that 

0  <   e'""  <  l/n-1   and   C   n  0'(e'")   =   0. 

Now  C  and  ^(e")  are  convex,  C  ['\  &ie")    contains  only  a  com- 
mon boundary  point,  and  ^^(e")  contains  an  interior  point.   Thus 
there  exists  a  hyperplane  separating  C  and  ^(e")  which  is  the 
desired  hyperplane. 

Theorem  3 »   Let  P  be  a  polyhedron  such  that  P  H  tY  =   0. 
Then  there  exists  a  hyperplane  separating  P  and  C   which  inter- 
sects &   only  at  the  origin. 

Proof:   Let  P  be  the  convex  hull  of  p,  ,...,p  .   Let  P'"*  be 
the  convex  cone  generated  by  p,,...,p  .   Then  P"~'  P  and  P"  is 
closed.   We  now  want  to  show  that  P"  C^  &  =   0.   Suppose  u  <£  P" 
and  u  &  C  where  u  7^  0.   Then  u  =  '"^iPi  +•••+  ^fV*-   where  X.  >  0 
and  >^  =  2Z  '^i  ^  ^*   Then  it  is  clear  that  [i/X    e  ff,    H.A  t  P  and 
[i/\  t   C  contradicting  the  hypothesis.   Thus  P"  H  0  =  0.   Then 
by  Theorem  2,  P"  and  6^' have  a  separating  hyperplane  which  meets 
0'  only  at  the  origin.   Clearly  this  hyperplane  separates  P  and 
C  as  desired . 
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§3*   Properties  of  vector  spaces  derived  from  the  separation 
theorem. 

In  this  section  we  will  derive  some  properties  of  vector 
spaces,  and  we  will  show  how  each  such  property  can  be  transla- 
ted into  a  theorem  on  matrices. 

Theorem  );.   Let  V  be  a  vector  subspace  of  E  .   Then  either 
1 .   V  f]   a  y^  0 
or        2.   V-i  i)  HO)    ^  0      , 

Proof:   '"'e  first  note  that  V  is  a  closed  convex  cone.   Now 
negating  (1)  we  have  V  T'l  (f  =   0 .      Thus  by  Theorem  2,  there  exists 
a  separating  hyperplane  (a,x)  =  C  such  that  (a,v)  <  0  for  v  &  V 
and  (a,s)  >  0  for  s  t  0',  s  /  0.   Now  v  e  V  implies  -v  t  V,  thus 
-(a,v)  =  {a,-v)  <  0,  hence  {'^,v)    >   0.   Thus  (a,v)  =  C  for  all 

V  £  V.   Thus  a  t  V"^'.   To  show  that  a  e  KC),  we  just  note  that 
{Q.,e^)    >   0  implies  a.  >  0  for  each  1,  thus  we  have  the  second 
alternative  . 

Definition.   A  vector  a  =  (a-,,..., a  )  will  be  called 
positive  if  a.  >   0,    i  =  l,...,n  and  ^  a.  >  0.   The  vector  is 
said  to  be  strictly  positive  if  a,  >  0  for  i  =  l,...,n. 

Corollary .   (Steimke's  Theorem)   Let  A  be  an  m  x n  matrix. 
Then  either 

(1)  there  exists  a  vector  v  such  that  vA  is  positive, 

or   (2)  there  exists  a  strictly  positive  vector  w  such 
that  Aw  =  '6\ 

Proof:   Let  V  =  A'e'".   Now  if  (1)  does  not  hold,  then 

Y  f\  Cf  =   d.   Thus  by  Theorem  b,,    V-^  f\   I({?')  f^   0.   Thus  there  exists 


«t.-      « 


I 
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a  strictly  positive  vector  w  such  that  (w,v)  =  0  for  all  v  e  V. 
Thus  (w,A'x)  =  0  for  all  x  fc  e"^.   Thus  (Aw,x)  =  0  for  all  x  t  e"'. 
Hence  Aw  =  0  which  is  the  second  alternative. 

V'e  now  note  the  "dual"  to  Theorem  [j ,  namely 

Theorem  5«   Let  V  be  a  vector  subspace  of  E^.   Then  either 
1.   V  n  liCf)    7^  0 
or       2.  V-'-  n  t  ^   o". 

Proof:   Applying  Theorem  ]\   to  V-*-  we  liave  the  result. 

Corollary .   Let  A  be  an  m  n  matrix.   Either 

1.   There  exists  a  vector  v  such  that  vA  is  strictly 
positive , 
or        2.   There  exists  a  positive  vector  w  such  that  Aw  =  0 . 

Proof:   Follol^I  similarly  to  the  previous  corollary. 

We  now  pursue  some  further  theorem.s  on  vector  spaces  and 
linear  inequalities  which  ;^-eneralize  the  preceding  results. 

For  this  purpose,  we  will  look  upon  E   as  being  the  pro- 

r   s         r 
duct  space  E  x  E  ,  where  E   is  the  r-dimensional  euclidean  space 

of  the  first  r  cartesian  coordinates  of  E^,  and  E   the  corres- 
ponding space  of  the  last  s  =  n-r  coordinates.   VJe  may  then 

look  upon  the  first  orthant  of  E  ,  which  we  now  denote  by  fr  ,  as 

r   s 

the  product  space  i^  x  C  •   '"'s  will  also  use  the  following  nota- 
tion:  If  u  =  (  U-,  ,  ,  .  .  ,\i  )  t  e\  then  u   '  =  ( u,  ,  .  .  .  ,u  )  and 
u  ^^  =  ( u  ^-,  ,  .  .  .  ,u  )  .   Furthermore,  if  S  c  e'^,  then 
S^^)  =  V'^^lu  &  S^  and  S^^)  =  ju^^^u^  ^\. 

Theorem_  6.   Let  V  be  a  vector  subspace  of  E^.   Let 
E^  =  E^'xE^,  (r  +  s=n)  and  fl'^  =  ff  -^   ;r^ .   Then  either 

(1)  [V  nc""]^^^  ^  0^'^ 
or       (2)  V-^  n  \U^^)  )cfr^]  ?^  0  . 


'v    -..1    ■' 
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Proof:   The  proof  will  proceed  by  induction  on  s.   For 
s  =  0,  the  statement  reduces  to  Theorem  l\..      We  now  assume  the 
theorem  to  be  true  for  all  s'  such  that  0  <  s'  <  s.   Applying 
the  induction  hypothesis  to  E''^   yE®~  ,  (where  E^   is  obtained 
by  "adjoining"  to  E^  any  one  of  the  last  s  coordinates)  we 
obtain  either 

or        (2)^^^:   v^  n  [I(a''"'^)x^^-^M  7^  0   . 

SupDOse  (2)  ,T  holds.   Then  noting  that 
- "        r+1  ^ 

we  have 

(2):   V-  f]  [K.o'^)  >.-^]  ^   0 

which  is  the  second  alternative.   Thus  we  may  assume  that  (l)j,+-i 
holds . 

Now  if  we  negate  (1),  we  have  [V  f\  o'^]  =^^^K      Then 

by  (1)^+1)  there  must  exists  P.  f.  V,  (j  =  l,...,s)  such  that 

P  .  =  ( u,  ,  .  .  .  ,u  ,u  ,,,...  ,u  ......  ,u  ,  )  t  0' 

J     1'    >    r'    r+i'    '  r+j      r+s 

and  (u-i  ,  .  .  .  ,u_,u   . )  7^  0     ■.   But  by  ne  eating  (1),  we  have 

■■^f  r)  S_ 

(u,  ,  .  .  .  ,u  )  =  0^'  ^   Thus  u   .  >  0.   Let  P  =  1_  P  ..   Then  P  e  V, 
P^^^  =  0^  ■'  and  P^''^  is  strictly  positive. 

Now  suppose  V^^'  i^  0'^^    j^'o^^K      (Note:   In  general 
y(r)  ^^  ^,(r)  ^  p^  p^  Ol^^'K)      Then  there  exists  v  :-   V  such  that 
^(r)  ^  ^Ar)    ^^^   ^(r)  ^  q^^'*.   Clearly  v+XP  e   V,  and  for  suffi- 
ciently large  X,    v+\P  etT.   Then  0^^^  7^  (v+AP)^^^  t  [V  r^rr]^^^ 


-.';'.,'. 


(  :  t. ..    ^ 
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which  contradicts  our  assumption  (namely  the  negation  of  (1)). 
Thus  V^^-  f^.O^^^    =t^^K      Hence,  by  Theorem  i|, 

(V^^^)^n  KC'^^'^)  ^   0.   Thus  there  exists  Z^^^  €  (V^^^f  such 

(  r) 
that  Z'  '  is  strictly  positive.   Let  Z  be  the  vector  obtained 

from  Z^^'    by  adjoining  s  zeros  to  Z    .   Then  clearly  Z  e  V   and 
Z  €  I{&^^'^)  y^chK      Thus  V^  n  [l{&^}>^a^]   4   0,  and  (by  negating 
(1))  we  have  shown  that  (2)  must  hold. 

Theorem  7«   ("Dual"  to  Theorem  6)   Let  V  be  a  vector  sub- 
space  of  S^.   Then  either 

(1)    [v^r,'^-"]^^^  7^  0^^-) 

or      (2)  V  n  [i(c;'')?<r:^]  4  o  . 

Froof :   Apply  Theorem  6  to  V  . 

The  following  corollaries  indicate  how  the  above  theorems 

translate  into  statements  concerning  systems  of  inequalities. 

Definition.   Let  x  =  (Xt,...,x  )  c  e"^.   Then  x  >  0  will 
1'    '  n 

designate  that  x  is  strictly  positive;  x  >  0  will  designate 
that  x  is  positive;  and  x  >  0  will  designate  that  x.  >  0, 
i  =  1, . . . ,n. 

Corollary.   (to  Theorem  6)   Let  A  and  B  be  matrices  of 
dimensions  m  «.  r  and  mxs  respectively.   Then  either 
(1)   there  exists  a  vector  u  such  that 

A'u  >  0   and   B'u  >  0 

or        (2)   there  exists  a  vector  x  >  0  and  a  vector  y  >  0 

such  that  Ax  +  By  =  0. 
(Note:   For  s  =  0,  this  is  Steimke's  Theorem.) 


li^ir^ijKc.-'   't- 
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I  0  '  ! .? ,-.     "  1 
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Proof:   Let  C  =  (A,B)  (hence  C  is  of  dimension  m  x(r+s)) 
so  that  C  =  (g!)-   Now  C'(e"^)  ^  Vc  f/ X  E^  .   Applying  Theorem  6 
to  V,  we  have 

either   (1')   [V  n  o'^^^'^]^^^  /'o^^^ 
or   (2')   V-^  (\  [lio"^)  X  Q^]    7^  0      . 
If  (1')  holds,  then  there  exists  v  e  V  such  that  v^^^  >  0  and 
V      '    >   0.   Now  there  is  some  u  €  e"^  such  that  C'u  =  v,  and  for 
this  u,  A'u  =  v^^^  and  B'u  =  v^^^  so  that  (1)  follows. 

If  (2')  holds,  then  there  exists  v  €  V^  such  that  v^^'^  >  0 
and  v^^^  >  0.   Then  for  all  u  £  e"^,  0  =  {C'u,v)  =  (u,Cv),  so 
that  Cv  =  d".   Thus 


-^ 


0  =  Cv  =  (A,B)(^,  J  =  Av^''^  +  Bv^^^ 

and  alternative  (2)  follows  in  this  case. 

Corollary.   (to  Iheorem  7)   (The  "dual"  of  the  previous 
corollary.)   Let  A  and  B  be  matiices  of  dimensions  mxr  and 
m  xs  respectively.   Then  either 

(1)  there  exists  a  vector  x  >  0  and  a  vector  y  >  0 
such  that  Ax  +  By  =  0 
or        (2)  there  exists  u  €  E   such  that  A'u  >  0  and  B'u  >  0. 
Proof:   Follows  from  the  alternatives  of  Theorem  7. 

§ I4. .   Derivation  of  Ff^rkas'  Theorem . 

We  now  illustrate  the  application  of  the  above  inequalities 
by  deriving  Farkas '  theorem  from  them. 

Theorem  8.   ( Inhomogeneous  form)   Let  B  be  an  m x s  matrix, 
let  b  be  an  s-dimensional  (column)  vector,  and  let  x  be  an 
m-dimensional  (column)  vector.   Let  ^   =  <ui  B'u-b  >  ol"  (note 


?  / 


I  . -:-.ii.t»:. 


that  ^  is  a  subset  of  e"^)  .  If  Jd  ^  0,    and  if  u  €  ^  implies 
(x',u)  >  p,  then  there  exists  y  >  0  such  that 
(1)   X  =  By 


and 


Let 


(2)   p  =  (y',b)   where   p  =  min  (x',u) 
Proof:   Let  x'  =  (x,  ,Xp,...,x  )  and  b'  =  {'b-.,hp,...,h    ) 


-x 


A  =.1 


-X 


my.  1 
\1- 


V 


1)^1 


and  let 


B  = 


my  s 


1  X  s 


Applying  the  corollary  (to  Theorem  7)  to  the  matrices  A  and  B, 

we  have  that 

either   (1')  there  exists  a  vector  £  >  0  and  a  vector  y  >  0  such 

that  Ar  +  By  =  0 
or      (2')  there  exists  u  c-  E^        such  that  A'u  >  0  and'B'u  >  0. 
^''e  now  show  that  (2')  cannot  hold.   For  suppose  (2')  held, 
then  there  exists  u  =  ( u,  ,  .  . . , u   .,  )  such  that  A'u  >  0  and 

X        til  ""X 

(m) 


^'u  >  0.   Let  u^"^-  =  {u.  ,..,  ,u)  .      Then  B'u"^  -  bu  ,,  >  0. 
—  ±  m  m+i  = 

A'u  >  0,  hence  u  _^-,    >  0  and  -(x'  u"^)  +  pu  ,,  >  0.   Let 

m+i  '      '■    m+1 


But 


^=    (uVu^^i).   Then 


I  ^ 


B'u  -  b  >  0 


and 


P  >  (x' ,u) 


1? 


1    .- 
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Thiis  u  e  y,   hence  p  -   mln  (x',u)  <  (x',u)  <  p  which  is  a  contra- 
diction.   Thus  we  conclude  that  (1')  holds. 

Thus  there  exists  E,   -    (^,,^p)  >  0  and  y  >  0  such  that 
A^  +  By  =  0,  where  ?t  +  ^p  ^  ^'      ^''®  "^^^  also  assume  that 
^]_  +  ^2  ~  -^  ^^y  dividing  through  by  2-,+£2)«   i'^ow 


— >  _ 

hence  -x?;2  +  By  =  0   and  £,  +  p£;p  -  b'y  =  0. 

— ?» 

^^'e  now  show  that  ^^  ^  Q) .      For  if  ^2  =  0,  then  By  =  0  and 

b'y  =  1.   Then  for  any  u  c^,  0  =  y'B'u  >  y'b  =  1  which  is  a 

contradiction.  Thus  £;p  >  0. 

Then  x  =  Bly/^^)  ^i^^  '^^.^^^Z^    +  P  =  t)'(y/£2)-   Let 

z  =  (y/?2^  ^^'^  -'-®^  ^"  '^  "^  ^®  such  that  p  =  (x',u").   Then 

p  =  (x',u'"')  =  z'B'u'"'  >  z'b  =  p  +  ^^/^_2_      ' 

Thus  4^  =  0  and  hence  ^^  =   1 .      Therefore  x  =  By  and 
P  =  (b'y)  =  (y',b). 

Corollary ♦   (Homogeneous  form)   Let  B  be  an  m x 3  matrix 
and  let  X  t  E^.      If  for  all  u  £  e"^  such  that  B'u  >  0  we  have 
(x',u)  >  0,  then  x  =  By  where  y  >  0. 


Proof:   In  Theorem  o,  set  b  =  0,  p  =  0  and  we  have  the 


result. 
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§ 5 •   An  application  to  a  dimension  relationship . 

As  a  further  illustration  of  the  separation  theorem,  we 
apply  it  to  a  fundamental  dimension  relationship  in  pame  theory. 
'''e  state  the  dimension  relationship  in  the  following; 

Theorem.   Let  TTj  and  TTtt  he  the  spaces  of  optimal  stra- 
tegies for  a  game  matrix  A.   Let  V^y  and  XItt  ^®  ^^^   smallest 
faces  of  the  simplices  of  admissible  strategies  which  contain 
TTj  and  TTtj  respectively.   Then 

where  D(S)  denotes  the  dimension  of  a  set  S. 

I'fe  will  not  give  a  proof  of  the  full  theorem  (see  [2]) 
but  shall  restrict  ourselves  to  an  essential  feature  of  the 
proof.   In  order  to  do  this  we  introduce: 

Definition .   A  vector  strategy  is  called  completely  mixed 
if  every  component  is  positive. 

Then  the  essential  feature  of  the  dinenslon  relationship 
is  imbedded  in: 

Theorem  9.   If  A  is  a  reduced  game  matrix,  then  there 
exists  a  completely  mixed  optimal  strategy  for  each  player. 

Proof:   Since  A  and  A+cl  have  the  same  optimal  strategies, 

we  may  assume  that  the  value  of  the  r^ame  v  =  0.   Let  R,  ,...,R 
J  o  1'    '  m 

be  the  rows  of  A,  and  C^,...,C   be  the  columns.   Then  (x,C.)  =  0 

J 

for  all  j  where  x  e  TTj  and  (y,F.)  =  0  for  all  1  where  y  f  TTtt  • 
Thus  if  X  e  TTt,  2 ^-if^-t  ~  0.   Let  P  be  the  convex  hull  of 

R, ,...,R  .   Then  0  £  F.   Wow  consider  any  convex  combination  of 
I'm  ^  "^ 

^l''**'^m  ^"^^  ^^^^  !>   x^R^  >   0.   This  implies  that  (x,  ,...,Xj^) 


-.••ijr;    1 


•.  .  i^-O:   '.  "•?!.'.■    { 


T^  .. 


■    It  •■/    t'. 


i  ■ 


I  !     r.,-,.- 


::-.*     )••> 
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is  optinal,  thus  equality  must  hold.   Hence  P  Pi  C^  =  0.   Thus  by 
the  separation  theorem  for  polyhedra,  there  exists  a  hyperplane 
(a,x)  =  0  such  that  (a,p)  <  0  for  p  e.  P  and  (a,s)  >  0  for  s  e  d', 
s  5^  0.   Letting  e.  be  the  i-th  unit  vector,  we  have  (a,e.)  >  0, 
hence  a.  >  0.   Thus  we  may  normalize  to  obtain  a  strategy  vector 
a'  which  is  completely  mixed.   But  (a',R. )  <  0,  hence  a'  £  TTjj» 
i.e.  a'  is  optimal  for  the  second  player. 

To  find  a  completely  mixed  optimal  strategy  for  the  first 
player,  we  need  only  to  apply  the  result  to  the  transpose  of  A. 
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